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Background of the Invention 



BACKGROUND OF THE INVENTION 
L Field of the Invention 

LQJL1 The present invention relates to a data storage system for storing data and a method for 
using a data storage system. 
^Description of the ft4e ?Related Art 

[02] With the current advances in information technology in a wide range of industrial fields, 
there is a need to provide electronic management of data using servers and data storage systems 
even in fields where electronic data management has nevernot been implemented. Even in 
fields where there us e d to be has been electronic data management using data storage systems, 
the amount of data is increasing significantly. As the amount of data increases, the storage 
capacity required also increases-atee. 

[0003] In such circumstances, it is not easy for data managers to »ew4^introduce new servers 
or data storage systems by their own , or to increase storage capacities at the right moment, to 
prevent crucial damages. And this has become too heavy a burden for them nowadays. damage. 
To solve such a this problem, athe business o f und e rtaking out-sourcing ef-data storage, such 
like lending as leasing servers or storaees storage. has been growing recently . (One example of 
such kind, for example s enterprise is called a data center business.) 
[ 000 4031 An example of this type of outsourcing business is disclosed described in 
Japanese Patent publication number 2000-501528 (which corresponds to t^SP U.S. P atent No. 
6,012,032), in which storage capacity is ten tleased and thea charge for data storage is collected. 
According to the invention, data storage devices are characterized as high-speed, 
medium-speed, and low-speed devices in proportion according to the access speeds of the 
devices. The accounting method for storage services according to this prior art involves 
requiring gjiigher price per unit gjlstorage capacity for data recording devices with higher 
access speeds^; i.e., ^charge for data storage is determined based on the type of data recording 
device being used in addition to the storage capacity being used. To collect the charge for data 
storage, information related to data elements ar e output is periodically sup plied from the data 
storage system-^each of the ehargerespective charges for high-speed storage devices, 
medium-speed storage devices, and low-speed storage devices arek calculated respectively , 
and jthen summed to collect the overall charge , p e riodically . 
Summary of the Invention 

BRIEF SUM MARY OF THE IN VENTION 

[ Means for solving the problems Q4] According to festhe prior art, the data storage devices 
are assigned and are fixed to each client according to the contract. Once a data storage device is 
assigned, the data remains in the device. 

[0006] However, while using this data storage system of the prior art , a sudden or a periodic 
increase efin traffic might can occur, and will cause degradation on degrading system 
performance. Such degradation engf system performance will occur 7 regardless of the capacity 
of the storage device. For example, even if there is enough free space, data access may be 
significantly delayed if there is too much access toward of specific data. 

[ 00071Th e obj e ct of the pres e nt OS] This invention is to provide provides a method for 

operating a data storage system^ in which the svstem performance of the data storag e system is 
kept at a fixed level during use of the data storage system. 

[0008] Anoth e r object of the The present invention is to provide an input means , which is u s ed 
te~se t for settin&jfae required data storage system performance JgYgl. 
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[ M o mis for solving th e problems}!!! order I0 O6I To solve the problems described above, a 
service -level guarantee contract is used for each client to guarantee a fixed service level related 
to storage performance. In the present invention, the data storage system is provided both with 
a performance monitoring part for monitoring th^operation status of the data storag e system 
and with a data migratin g transfer means. 

[0010] The performance monitoring part includes-^ a part for setting performance requirement 
parameters for various elements such itkefyi device busy rate, data transfer speed and so on^ that 
defines d efine storage performance. Performance The performance requirement parameter 
represents a desired storage performance.-Sueh JIhis parameter can be, for example, a 
threshold, a function, and so on. 

[071 The performance monitoring part also includes^ a monitoring part for monitoring actual 

storage performance variables that change according to the operation status of the data storage 

system. If the-monitoring of the parameters indicates a drop in the-storage performance in a 

specific logical device or the entire data storage system, the data migratin g transfer means 

inierate stransfers data so that load is distributed. 

Brief Description of th e Drawings 

BRIEF DES CRIPTION O F THE DRAWINGS 

[ 081 Fig. 14A is a schematic drawing of the RAID group. 

Af091 Fig. 2 is a schematic drawing illustrating the relationship between the data center, 
providers, and the client PCs (end-user terminals). 

[ 101 Fig. 3^A is a detailed drawing of a data storage system provided with a performance 
monitoring part m onitor. 

Af 111 Fig. 4 is a flowchart of the operations used to set a service level agreement (SLA). 
[ 12] Fig. 5]Ah is an SLA category selection screen serving as part of a user interface for setting 
an SLA. 

[ 131 Fig. 6}AJjy| performance requirement param e ter setting screen serving as part o f a user 
interface for setting an SLA. 

[141 Fig. 7]Aft is an example of a disk busy rate monitoring screen. 
[151 Fig. 8]A is a flowchart of the operations used to migrate data. 

[ 16] Fig. 9j A is a flowchart of the operations used to create a data storage system operating 
status report. 

[ 171 Fig. 104 A is a schematic drawing of a-data migration toward anothe r to a device^ outside 
the data storage system. 

[ 181 F ig. 1 1 ] A sample is an example of a performance monitoring screen. 
[ 191 Fig. \2}Aft is an example of a performance monitoring table. 

[ 201 Fig. 13jAn is an example of a performance monitoring table containing post-transfer 
prediction values for aft e r th e migration operation . 
Description of the Preferred Embodiments 

DESCRIPTIO N OF THE PREFERRED EMBODIMENTS 

[211 The embodiments of the present invention will be described in detail with 
ref e renc e s reference to the figures. 

[0012] Fig. 2 shows the architecture of a network system including a data center (240) 
according to an embodiment of the present invention and client PCs accessing the data center 
(2 4 0). 240. In this figure, the data center (240) consists of the elements shown below the 
LAN/WAN (local area network/wide area network-304> ZM^Client PCs (201-203) access the 
data center (240) via the LAN/WAN (204) to receive various services provided by from 
providers A-C £233- 235^ . 235. Servers (205-207) and data storage systems (209) are 
connected to a storage area network ( S AN 208) v )_208 r 
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[004^22] Fig. 3 shows the detail of the internal architecture of tkenstorage system (209). 
202^_Different types of storage media are stored in the storage system (209). 20SLJ n this figure, 
types A, B and C are exemplary and shown for easy t o_ajd understanding. The number of 
storage media types does not have to be can be dif ferent than three, and can be varied). 
[004422] The storage unit includes a service processor SW (325), (SVP) 325. that 
monitors the performance of these elements and controls the condition settings and execution 
of various storage operations. The SVP (325) is connected to a performance monitoring PC 
(333), 

fOQ^54 323^ The performance maintenance described above is provided in the present invention 
by using a performance monitoring part (324) in the form of a program running on the SW 
£^25 VSVP 325. More specifically, performance maintenance is carried out by collecting 
parameters that quantitatively indicate performances the performance of individual elements. 
These collected parameters are compared with performance requirement parameters (326). 326. 
The performance requirement parameters (326) are set in the-SVP (325) of the data storage 
system. Depending on the results of the comparison between the actual storage performance 
variables and performance requirm g requirement parameters/performance maintenance 
operations will be started. are initiated. This process will be described in detail later along with 
the§ description of service level agreements. In addition to simple comparisons of numerical 
values, the comparisons with performance requirement parameters can include comparisons of 
flexible conditions such as comparisons with functions. 

[ 0016 241 Since the SVP (325) is set inside the data storage system, it can be used only by 
the administrator. Thus, if functions similar to those provided by the-performance monitoring 
part (324) are to be used from outside the data storage system, this can be done by using the 
performance monitoring PC. In ot h er words, in the implementation of the present invention, 
the location of the performance s torage part m pjlitor does not matter. The present invention can 
be implemented as long as data storage system performance can be monitored, comparisons 
between the actual storage performance variables and performance requiring parameters can be 
made, and the data storage system can be controlled based on the comparison results. 
[ 0017 251 The following is a more specific description. First, examples of parameters 
monitored by the-performance monitoring part (324) will be described. Examples of 
parameters includes disk free space ratet a disk busy rate? * I/O accessibility^ data transfer 
volume^ data transfer speedy and the amount of cache-resident data. The disk free space rate is 
defined as (overall contracted disk space) divided by (free disk space). The disk busy rate is 
defined as the time during which storage media (the physical disk drives) are being accessed 
per unit time. I/O accessibility is defined as the number of read/write operations completed per 
unit time. Data transfer volume is defined as the data size that can be transferred in one I/O 
operation. Data transfer speed is the amount of data that can be transferred per unit time. And 
the amount of cache-resident data is the data volume being staged to the cache memory. 
[ 0018 261 While u s ing the data storage system is in use , storage performance can fall if the 
number of accesses to a specific device suddenly increases or increases during specific times of 
the day. Reduced storage performance can be detected by checking tfwhether the parameter 
values described above exceed threshold values. If this happens, the concentrated load 
against on some specific device is distributed so that a required storage performance can be 
maintained. 

[ 001 9]when 271 When storage performance falls due to localized concentration of 

accesses, the accesses must be distributed to maintain storage performance. 
[0020] The present invention provides a method for distributing storage locations for data in a 
data storage system. 

[ 0021 281 In the network system shown in Fig. 2, the-data center (240) equipped with the 
data storage system (209) and the-servers (205-207) is contracted to provide storage capacity 
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and specific servers to the-providers £233- 235). Th e providers ( 235 . Providers 233-2354 use 
the storage capacities allowed by their respective contracts and providoG Provide various 
services to end-users' client PCs £201-203) via the LAN/WAN. Thus, this network system is 
set up through contracts between three parties (data center - provider contracts and provider - 
end user contracts). 

r 0022 2£1 Fig. 2 also schematically- shows the schematic relationship between the data 
center £240) equipped with the data storage system and the servers, the providers 
(233-235)^22^ and the client PCs £201-2©3)r-2m^The end user uses a client PC (201-203) to 
access the-data center £240) via a network. The dat a_ _Data center (240) stores data of the 
providers (233-235) contracted by the end user. Th e provid e rs ( Providers 233-2354 entrust the 
management of the data to the-data center (2 4 0) and the data center f2 4 0) 240, which charges 
the-fees to the-providers £233- 235). 235. The client using the services provided by the 
providers pays the charge for such services. 

[0 023 301 As described above, the provider enters into a contract with the data center for 
system usage. The performance of the hardware provided by the data center ( performance of 
the data storage system, servers, and the like) is directly related to the quality of the services 
provided to clients th eby a provider. Thus, if a guarantee that storage performance will be 
maintained can be included in the contract between the data center and the provider, the 
provider will be able to provide services withgf reliable quality to the end users. The present 
invention makes this type of reliability in service quality possible. 

[002421] A concept referred to as tfeeg service level agreement (SLA) is introduced in the 
data center operations that use this network system. The SLA is used for quantifying storage 
performance that can be provided by the-data storage system (209) andjgr providing 
transparency for the services that can be provided. 

[ 0025 321 Service level agreements (SLA) will bo are now described briefly. In service 
contracts, it would be desirable to quantify the services provided and to clearly identify service 
quality by indicating upper bounds or lower bounds. For the party receiving services, this has 
the advantage of allowing easy comparisons with services from other firms.^Ak e In addition , 
services that are appropriate to the party's needs can be received at an appropriate price. For the 
provider of services, the advantage is that, by indicating the upper bounds and lower bounds 
that can be provided for services and by clarifying the scope of responsibilities of the service 
provider, clients receiving services are not likely to hold unrealistic expectations and 
unnecessary conflicts can be avoided when problems occur. 

[ 0026 331 Of the agreements between the data center, the provider, and the end user, the 
service level agreement (SLA) ing f the present invention relates to the agreements between the 
data center and the-providers £233- 235). 235. The service level agreement is d e ter 
fflrffleddetermined by the multiple elements to be monitored by th«-performance monitoring part 
(324) described above and the storage device contract capacity (disk capacity) desired by the 
provider. 

[ 0027 341 The following is a description of the flow of operations performed when the 
data center and a provider enter into a service level agreement using these parameters. 
[0028] First, the flow of operations performed to determine the contents of the guarantee 
(target performance) given by the data center to the provider will b e j j> described using Fig. 4. 
(Flowchart for setting service level agreement: step 401 -step 4 07 407. )t 
[ 00291In 351 At step 402 of Fig. 4, the provider selects one of the storage guarantee 
categories for which the data center wants a guarantee, e.g., disk busy rate by RAID group (rate 
of time during which storage medium is active due to an access operation), proportion of free 
storage space (free space / contracted space) (step 4 02) . The operations p e rformed for entering 
a setting in the selected category will be described later using Fig. 5. 
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[003 0361 Next, the provider sets guarantee contents and values (required performance 
levels) for the selected guarantee categories (step 403). For example, if the guarantee category 
selected at step 402 is the drive busy rate, a value is set for the disk busy rate, e.g., "keep 
average disk busy rate at 60% or less per RAID group" or "keep average disk busy rate at 80% 
or less per RAID group." If the guarantee category selected at step 402 is the available storage 
capacity rate, a value is set up for that category, e.g., "increase capacity so that there is always 
20% available storage capacity^ (In other words, disk space must be added if the available 
capacity drops below 20% of the contracted capacity. If the capacity contracted by the 
provider is 50 gigabytes, there must be 10 gigabytes of unused space at any time) 11 . \Jn these 
examples, "60%" and "80%" are the target performance values (in other words, agreed service 
levels). 

[ 0031 371 Once the guarantee categories and guarantee contents have been determined, 
the charge for data storage associated with this information is presented to the provider. The 
provider decides whether or not to accept these charges (step 404). Since the guarantee values 
contained in the guarantee contents affect the usag euse of hardware resources needed by the 
data center to provide the guarantee contents, the fees indicated to the provider will vary 
accordingly. Thus, the provider is able to confirm the variations in the charge. Also, if the 
charge is not reasonable for the provider, the provider can reject the charge and go back to 
entering guarantee content information. This makes budget management easier for the 
provider.-Ste fr__Stens 403 and-step 404 will be described later using Fig. 6. 
[38] Next, all the guarantee categories are checked to see if guarantee contents have been 
entered (step 405). Once this is done, the data center outputs the contracted categories again so 
that the provider can confirm guarantee categories, agreed service level (performance values), 
the charge, and the like (step 406). It would bek desirable to let the provider confirm the total 
charge for all category contents as well. 

1 0032 391 Fig. 5 is a drawing for the purpose of describing step 402 from Fig. 4 in detail. 
As shown in Fig. 5, guarantee contents can, for example, be displayed as a list on a PC screen. 
The provider, i.e., the data center's client, makes selections from this screen. This allows the 
provider to easily select guarantee contents. If the provider has already selected the needed 
categories, it would be desirable, for example, to have a control flow (not shown in the figure) 
from step 402 to step 406 in Fig. 4. 

[ 0033 401 Fig. 6 shows an exemplificd exmnnlary method for implementing step steps 403 
and step 404 from Fig. 4. In Fig. 6, recommended threshold values and their fees are displayed 
for different provider operations. For example, provider operations can be divided into type A 
(primarily on-line operations with relatively high restrictions on delay time), type B (primarily 
batch processing with few delay time restrictions), type C (operations involving large amounts 
of data), and the like. Suggested drive busy rates corresponding to these types would be 
displayed as examples. Thus, the provider can choose which type its end-user services belong 
to and can select the type. The values shown are recommended values, so the provider can 
modify these values later based on storage performance statistics data presented by the data 
center. The method indicated in Fig. 6 is just one example, and it would also be possible to 
have stee stens 403 and step-404 provide a system where values simply indicating guarantee 
levels are entered directly and corresponding fees are confirmed. 

[ 0034 411 As described above, with referonces reference to Fi gFigs . 4 through-Fig. 6, the 
operations performed for determining service guarantee categories and contents are practiced. 
The selected service guarantee categories and contents are stored in storage means, e.g., a 
memory, of the SVP via input means of the SVP. This information is compared with actual 
storage performance variables collected by the monitoring part. Storage is controlled based on 
these results. R e garding th e JFor entry of service categories and content performance target 
values into the SVP, the n ee d to use of the SVP input means of th e S VP can be eliminated by 
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inputting the information via a communication network from a personal computer supporting 
the steps in Fig. 4. 

r 0035 421 Fig. 4 shows the flow of operations performed for entering a service level 
agreement. Fig. 5 and Fig. 6 show screens used by the provider to select service levels. The 
category selection screen shown in Fig. 5 corresponds to step 402 fremgf Fig. 4 and the 
threshold value settings screen corresponds to step 403 #amof Fig. 4. 

[" 0036 431 The service level agreement settings are made with th e following st e ps. Thea s 
follows. A provider wantin g desiring a contract with the data center selects one of the 
categories from the category selection screen shown in Fig. 5 and clicks the corresponding 
check box (step 402 of Fig. 4). A threshold setting screen (Fig. 6) for the selected category is 
displayed, and the provider selects the most suitable option based on the scale of operations, 
types of data, budget, and the like. The threshold is set 7 by checking one of the checkboxes on ? 
as a screen such as the one shown in Fig. 6 (step 403 of FijU-4) . 

[ 0037 44] The following is a description of a method for operating the data center in order 
to actually fulfill the service level agreement mad ecreated by the process described above. 
[0038] Fig. 7 shows a sample busy rate monitoring screen. Busy rates are guaranteed for 
individual RAID groups (described later). The busy rate monitoring screen can be accessed 
from the SVP (325) or the performance monitoring PC (323). 323. The ttsaeeuse status for 
individual volumes is indicated numerically. The busy rate monitoring screen includes^ a 
logical volume number (701); an average busy rate (702) for the logical volume; a maximum 
busy rate (703) for the logical volume; a number identifying a RAID group, which is formed 
from multiple physical disk drives storing sections of the logical volume; an average and 
maximum busy rate for the entire RAID group (706); and information (704, 705) indicating the 
usageuse status of the RAID group. Specific definitions will be described later using Fig. 1 1 . 
r 00391The information (4 51 Information 704, 705) indicating RAID group usage status w*H 
b eis now described. A RAID group is formed as a set of multiple physical disk drives storing 
multiple logical volumes that have been split , including the volume in question. Fig. 1 shows 
a sample RAID group formed from three data disks. (The number of disks does not need to be 
three and can be varied).) In this figure, RAID group A is formed from three physical disk 
drives D1-D3 storing four logical volumes VQV0-V3. In this example, the new RAID group A 
is formed from the logical volumes VI- V3 without logical volume V0. 
[ 00 4 01The information ( 461 Information 704, 705) indicating RAID group usage status for 
the logical volume VOV0 is information indicating the overall busy rates for the newly formed 
RAID group A' (RAID group A without the-logical volume VQ VO) . The numeric values 
indicate the average (704) and-the maximum (705) busy rates. In other words, when the-logical 
volume VQVO is moved to some other RAID group, the values indicate the average drive busy 
rate for the remaining logical volumes. 

[ 0041 471 After the service level agreement has been set, a performance requirement 
parameter, titee such as threshold values-are, is set based on the service level agreement, and the 
relationship between actual storage busy rates (702 -705) and the threshold values are 
monitored continuously monitored through the monitoring screen shown in Fig. 7. Data is 
mi grated transferred automatically or by an administrator if a numerical value indicating the 
actual storage performance variable (in this case, the busy rate) is about to exceed an "average 
XX%" value or the like guaranteed by the service level agreement^; i.e., the value exceeds the 
performance requirement parameter, such as the threshold value. (The "average XX%" 
guaranteed by the service level agreement is generally set in the-performance monitoring part 
(324) as the threshold value, and the average value is kept to XX% or less by moving data when 
a parameter exceeds the threshold value.) 

[481 The following is a detailed description of a method for guaranteeing a drive busy rate. 
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[004242] First, ttsm gwith reference to Fig. 1, the relationship between logical volumes 
(logical devices), which the server uses as storage access units, and physical drives, in which 
data is recorded, will be i§ described. Taking a data storage system with a RAID (Redundant 
Array of Inexpensive Disks) Level 5 architecture as an example, multiple logical volumes are 
assigned to multiple physical drives (RAID group), as shown in Fig. 1. The logical volumes 
are assigned so that each logical volume s volume is distributed across multiple physical drives. 
This data storage system is set up with multiple RAID groups, each group being formed from 
multiple physical drives. Logical volumes, which serve as the management units when 
recording data from a server, are assigned to these RAID groups. RAIDs RAIDS and RAID 
levels are described in D. Patterson, G. Gibson, and R. Katz, "Case for Redundant Arrays of 
Inexpensive Disks (RAID)," Report No. UCB/CSD 87/391 (Berkeley: University of California, 
December 1987). In Fig. 1, the RAID group is formed from three physical drives D, but any 
number of drives can be used. 

[ 00 4 3 501 With multiple logical volumes assigned to multiple physical drives as described 
above, concentrated server accesses teof a specific logical volume will negatively affect other 
logical volumes associated with the RAID group to which the specific volume is assigned. Aise 
Moreover , if there is an overall increase in accesses to the multiple logical volumes belonging 
to a RAID group, the busy rate tegf the physical drives belonging to the RAID group will 
increase, and the access delay time for the logical volumes will quickly increase. The busy rate 
for the RAID group can be kept at or below a specific value by monitoring accesses to these 
logical volumes, collecting statistical data relating to access status tegf drives, and moving 
logical volumes to other RAID groups with lower busy rates. 

[ 0014 511 If the agreement between the provider and the data center involves keeping the 
busy rate of the physical drives of a particular RAID group at or below a fixed value, the data 
center monitors the access e s access status of such RAID group in the data storage system and 
moves the logical volume in the RAID group to another RAID group if necessary, thus 
maintaining a performance value for the provider. 

[ 0045 521 Fig. 1 1 shows an example of a performance management table used to manage 
RAID group 1 performance. Performance management tables are set in association with- 
individual RAID groups in the data storage system and are managed by the performance 
management part in the SVP. In this table, busy rates are indicated in terms of access time per 
unit time for each logical volume f VO. VI. VP. VL V2, ...) in each drive (©IdlL D2, D3) 
belonging to the RAID group 1. For example, for drive Dl in Fig. IMkll. the busy rate for the 
logical volume VOV0 is 15% (1 5 seconds out of the unit time of 100 seconds is spent accessing 
the-logical volume VQVO of-4he drive D 1), the busy rate for the-logical volume V4-V1 is 30% 
(30 seconds out of the unit time of 100 seconds is spent accessing the-logical volume V4-V1 of 
the drive Dl), and the busy rate for the-logical volume V2 is 10% (10 seconds out of the unit 
time of 100 seconds is spent accessing the-logical volume V2 of4he drive D 1). Thus, the busy 
rate for drive Dl (which is the sum of the logical volumes per unit time) is 55%. Similarly, the 
busy rate for drive D2 isr 10% for tha-logical volume V&jCL 20% for the-logical volume Vir Vl, 
and 10% for the-logical volume V2. The busy rate for4he drive D2 is 40%. Similarly, the busy 
rates for the-drive D3 arer 7% for the-logical volume VQ fVO, 35% for the-logical volume V+fJ^ 
and 15% for the-logical volume V2. The busy rate for the-drive D 2 [?D 3?} is 57%. Thus, the 
average busy rate for the three drives is 50.7%.^Also Furthermore, the maximum busy rate for 
a drive in the RAID group is 57% (drive D3). 

[0046531 Fig. 12 shows an example in which a-logical volume V3 and-a logical volume 
V4 are assigned to RAID group 2. In this example, drive Dl has a busy rate of 15%, drive D2 
has a busy rate of 1 5%, and drive D3 has a busy rate of 10%. The average busy rate of the 
drives belonging to the RAID group is 13.3%. 
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[0044 541 These drive busy rates can be determined by having the DKA of the disk control 
device DKC measure drive access times as the span between ^drive access request through the 
response from the drive, and reporting these times to the performance monitoring part. 
However, if the disk drives themselves can differentiate accesses from different logical 
volumes, the disk drives themselves can measure these access times and report these times to 
the performance monitoring part. The drive busy rate measurements need to be performed 
according to definitions within the system so that there are no contradictions. Thus, definitions 
can be set up freel yfinely as long as the drive usage use status can be indicated according to 
objective and fixed conditions. 

[ 00 4 8 551 In the following example, an average drive busy rate of 60% or less is 
guaranteed by the data center for the provider. If the average drive busy rate is to be 60% or 
less for a RAID group, operations must be initiated at a lower busy rate (threshold value) since 
a delay generally accompanies an operation performed by the system. In this practice, if the 
guaranteed busy rate in the agreement is 60% or less, operations are begun at a busy rate 
(threshold value) of 50% to guarantee this required performance. 

[ 00 4 9 561 In Fig. 1 1 described previously, the average busy rate of the drives in the RAID 

group exceeds 50%, making it possible for the average busy rate of the drives in the RAID 

group 1 to exceed 60%. The performance monitoring part of the SVP therefore 

mi grates tran sfers one of the logical volumes from the-RAID group 1 to another RAID group, 

thus initiating operations with an average drive busy rate in the-RAID groupj. that is 50% or 

lower. 

[ 0050 57] In this case, two more issues must be dealt with to begin operations. One is 
determining which logical volume is to be mi grated transferred from the-RAJD group 1 to 
another RAID group. The other is the RAID group to which the volume is to be 
mi grated transferred . 

[ 0051 581 In migratin g transferrin g a logical volume from the-RAID group 1, the logical 
volume must be selected so that the source group, i.e., the-RAID group 1, will have an average 
busy rate of 50% or less. Fig. 1 1 also shows the average drive busy rates in the-RAID group 1 
when a volume is migrated transferred to some other RAID group. In this example, if the 
logical volume VOV0 is migrat e d transferred to some other RAID group, the average drive 
busy rate from the remaining volumes will be 40% (corresponds to the change from RAID 
group A to A' in Fig. 1). Migrating the JTransferrin g logical volume VI to some other RAID 
group results in an average drive busy rate of 22.3% for the remaining volumes. Migrating the 
Transferring logical volume V2 to some other RAID group results in an average drive busy rate 
of 39.0% for the remaining volumes. Thus, for any of these the rate will be at or below 50%, 
and any of these options can be chosen. In the description of this embodiment, the~logical 
volume V2 is migrated transferred , providing the lowest average busy rate for tfee-RAID group 
1. In addition to reducing the average busy rate to 50% or lower, the logical volume to 
migrate b e transferred can also be selected on the basis of the frequency of accesses since 
migrating transferring a logical volume experiencing fewer accesses will provide less of an 
impact on accesses. For example, in the case of Fig. 11, the-logical volume VQVO can be 
selected since the average busy rate is lowest. Alternatively, since migratin g txansfemng 
logical volumes that contain less actual data will take less time, it would be possible to keep 
track of data sizes in individual logical volumes (not illustrated in the figure) and to select the 
logical volume with the least data. 

[ 00521 £] Next, the destination for the logical volume must be determined. In determining 
a destination, not violating the agreement with the provider requires that the current average 
drive busy rate stays at or below 50% 4 and the destination RAID group for the selected logical 
volume must have an average drive busy rate that stays at or below 50% (the threshold value) 
even after the selected logical volume has been moved there. Fig. 13 shows a prediction table 
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for when the logical volume VI is moved from the-RAED group 1 to fee-RAID group 2. The 
average drive busy rate of the RAID group 2 is currently 13.3%, so i^the group can accept a 
logical volume from another RAID group. The table shows the expected drive busy rates for a 
new RAID group, formed after receiving logical volume ViVl (bottom of Fig. 13). As shown 
in the table, the predicted average drive busy rate after accepting the new volume is 41.7%, 
which is below the threshold value. Thus, it is determined that the volume can be accepted, and 
the formal decision is then made to move the-logical volume V4Y1 from the-RADD group 1 to 
the-RAID group 2. To guarantee performance in this manner, it is necessary to guarantee the 
busy rate of the source RAID group as well as ^calculate, predict, and guarantee the busy rate 
of the destination RAID group before moving the logical volume. If the expected busy rate 
exceeds 50%, a different RAID group table is searched and the operations described above are 
repeated. 

r 0053 601 As described above, the data center can provide the guaranteed service level for 
the provider in both the logical volume source and destination RAID groups. 
[ 005 461] In the example described above, a 50% threshold value is used for 
migratin g transferrin g logical volumes and a 50% threshold value is used for receiving logical 
volumes. However, using the same value for both the migratin gt ransfer condition and the 
receiving condition may result in logical volumes being migrated repeatedlyr transferred. Thus, , 
it would be desirable to set the threshold for the migratin g transfer condition lower than the 
threshold for the receiving condition. 

[ 0055 62] Also, the average busy rates described above are used here to indicate the busy 
rates of drives in aJRAID group. However, because the drive with the highest busy rate affects 
responses for all accesses to jlRAID group, it would also be possible to set the guarantee 
between the provider and the data center based on a guarantee value and ^corresponding 
threshold value for the drive with the highest busy rate. 

1" 0056 631 Furthermore, the performance of the drives in the-RAID group 1 (source) and 
the performance of the drives in4he RAID group 2 (destination) are presented as being 
identical in the description of Fig. 13. However, the performance of the drives in the 
destination RAID group 2 may be superior to the performance of the source drives. For 
example, if read/write speeds to the drive are higher, the usage time for the drives will be 
shorter. In such cases, the RAID group 2 busy rate after receiving the logical volume can be 
calculated by multiplying a coefficient reflecting performance differences t eamong the busy 
rates of individual drives of the-logical volume ¥4 VI in~the RAID group 1 in rel ation to the 
busy rates of individual drives in the RAID group 2. If the destination drives have inferior 
performance, inverse coefficients can be used. 

[ 0057 641 In the operation described above, the performance management part (software) 
can be operated with a scheduler so that checks are performed periodically and operations are 
performed automatically if a threshold value is exceeded. However, it would also be possible 
to have the administrator look up performance status tables and expectation tables to determine 
if logical modules should be mi grat e d. transferred. If a m i gra t i on trans fer is determined to be 
necessary, instructions for niigratin g transferring the logical module are sent to the data storage 
system. 

[ 0058 651 In the example described above, the RAID groups have the same guarantee 
value. However, it would also be possible to have categories such as type A, type B, and type C 
as shown in Fig. 3, with a different value for each type based on performance^ e.g., type A has 
a guarantee value of 40%; type B has a guarantee value of 60%; type C has a guarantee value of 
80%. In this case, logical volumes would be migrated transferred between RAID groups 
belonging to the same type. 

[005^ 661 This concludes the description of the procedure by which performance 
guarantees are set through a service level agreement and of an example of how performance is 
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guaranteed using busy rates of physical disk drives. Next, the procedure by which a service 
level agreement is implemented in actual operations will be described with reference to Fig. 
using an example in which performance is guaranteed by moving data. 
[ 0060 671 At the start of operations or at appropriate times, threshold values for 
parameters are set up manually for fee-performance monitoring part 324 on the basis of 
performance requirement parameters guaranteed by the service level agreement (step 802). 
The performance monitoring part detects when actual storage performance variables of the 
device being monitored exceed or drop below threshold values (step 803, step 804). Threshold 
values are defined with maximum values (MAX) and minimum values (MEN). The variable 
exceeding the maximum value indicates that it will be difficult to guarantee performance. The 
variable about to drop below the minimum value indicates that there is too much extra 
availability in resources so that the user is operating beyond specifications (this will be 
described later). If the variable exceeds the threshold value in the form of an average value 
XX%, a determination is made as to whether the problem can be solved by 
migratin g transferring data (step 805). As described with reference to R gFigs . 1 1 through Fig? 
14, this determination is made by predicting busy rates of the physical drives belonging to the 
source and destination RAID groups. If there exists a destination storage medium exists that 
allows storage performance to be maintained, data will be migrated transferred (step 807). This 
data mi gratin ^ transfer operation can be performed manually based on a decision by an 
administrator, using server software, or using a micro program in the data storage system. If no 
destination storage medium is available because the maximum performance available from the 
data storage system is already being provided, the-SVP 325 or the-performance monitoring PC 
323 indicates this by displaying a message to the administrator, and notifies the provider^ if 
necessary. The specific operations for migratin g transferring data can be provided by using the 
internal architecture, software, and the like of the data storage system described in Japanese 
patent, publication number 9-274544. 

[ 0061 681 Fig. 9 shows the flow of operations for generating reports to be submitted to the 
provider. This report contains information about the operation status of the data storage system 
and is sent-periodically sent to the provider. The operation status of the data storage system can 
be determined through various elements being monitored by the-performance monitoring part 
324. The performance monitoring part collects actual s t or age storage performance variables 
(step 902) and determines whether the performance guaranteed by the service level agreement 
(e.g., average XX% or lower) is achieved or not (step 903). If the service level agreement 
(SLA) is met, reports are generated and sent to the provider periodically (step 904, step 906). If 
the service level agreement is not met, a penalty report is generated and the provider is notified 
that a discount will be applied (step 905, step 906). 

\QQ6269] This concludes the description of how, when the busy rate is about to exceed the 
performance requirement parameters which indicates indicate the agreed service level^-mof the 
contract due to accesses concentrated in a localized manner (to a specific physical drive), the 
logical volumes belonging to that physical drive are migrated transferred so that the accesses to 
each physical drives are equalized. An alternative to the method described above for 
deconcentrating localized load concentration in a data storage system is to temporarily create a 
mirror disk of the data for which the load that is concentrated (in the example shown in Fig. 7, 
the data having a high busy rate) 4 so that accesses can be deconcentrated, thus maintaining the 
performance guarantee values. This method must take into account the fact that, on average, 
half of the accesses to the mirrored original drive will remain. In other words, post-mirroring 
busy rates must be predicted by taking into account the fact that accesses corresponding to half 
the busy rate of the logical volume will continue to be directed at the current physical drive. 
[ 0063 70] In the practice as described above, in the load deconcentration method 
involving the migratin gt ransfer of data (to maintain performance guarantee values) as 
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described above, the data (a logical volume) will b o migrat e d are transferred to a physical drive 
in a different RAID group within the same data storage system. However, as shown in Fig. 10, 
the data can also be m i grated trans ferred to a different data storage system connected to the 
same storage area network (SAN). In such case, it would also be possible to have devices 
categorized according to the performance itthev can achieve, e.g., -a device equipped with 
high-speed, low-capacity storage devices-^ or "a device equipped with low-speed, 
high-capacity storage devices-. When determining a destination (in another device), the 
average busy rates and the like for the multiple physical drives in the RAID group in the 
different data storage system are obtained and used to predict busy rates at the destination fez 
once the logical volume has been migrated. transferred. These average busy rates 4 and the like., 
of the multiple physical drives in the other device can be obtained by periodically exchanging 
messages over the SAN or issuing queries when necessary. 

[0064 711 The service level agreement made between the provider and the data center is 
reviewed when necessary. If the service level that was initially set results in surplus or 
deficient performance, the service level settings are changed and the agreement is updated. For 
example, in Fig. 6, the agreement may include "XX > YY > ZZ" and a physical drive is 
contracted at YY%, the average type B busy rate.4t If, in this case, the average busy rate is 
below ZZ%, there is surplus performance. As a result, the service level is set to type C average 
busy rate of ZZ% and the agreement is updated. By doing this, the data center can gain free 
space, s o as to provide them to for a new potential customer, and the provider can cut cost. And 
this This is beneficial to both e£4he-parties. 

[ 0065 721 A s anothe r Another example of a service level agreement74ke*=e is a type of 
agreement that in which the service level will be temporarily changed t e mporary . For example, 
a provider may want to propose a newspaper advertisement concerning som e particular ^ 
contents stored in a particular physical disk drive. In such this case, if such contents are stored 
in a high-capacity, low-speed storage device, they have to be moved to a low-capacity, 
high-speed storage device, as a flood of data access is expected? because of the advertisement. 
In this case,j|i| additional charge for using the high- speed storage device will be paid.-As 
Because the increase of data-access to such data wiHjg expected to be a temporal one temporarv , 
the provider may want the concerning dat a concerned to be stored in the low-capacity, 
high-speed storage device for -s omog short period? and then moved back to the high-capacity, 
low-speed storage device to cut expense. The data center will be notified in advance^ that the 
provider wants to modify the service level agreement for the particular data. Then, during 
thisthe period specified by the provider, the data center will modify the performance 
requirement parameter for the specified data. 

r 0066 731 In the description above, busy rates of physical drives in RAID groups are 
guaranteed. However, services based on service level agreements can be provided by meeting 
other performance guarantee categories, e.g., the rate of free disk space, I/O accessibility, data 
transfer volume, and data transfer speeds. 

[ 0067 741 For example, a service level agreement may involve allocating 20% free disk 
space at any time, relative to the total contracted capacity. In this case, the data center leasing 
the data storage system to the provider would compare the disk capacity contracted by the 
provider with the disk capacity that is actually being used. If the free space drops under 20%, 
the provider would allocate new space so that 20% is always available as free space, thus 
maintaining the service level. 

[ 0068 75] In the embodiment described above and in Fig. 3, the server and the data storage 
system are connected by a storage area network. However, the connection between the server 
and the data storage system is not restricted to a network connection. 
[0069} 
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[Advantages of th e i.nv e ntion1 761, As described above, the present invention allows the-data 
storage locations to be optimized according to the operational status of theg data storage system 
and allows loads to be equalized when there is a localized overload. As a result, data storage 
system performance can be kept at a fixed level guaranteed by an agreement even if there is a 
sudden increase in traffic. 



